Raman image-activated cell sorting.

The advent of image-activated cell sorting and imaging-based cell picking has advanced our knowledge and exploitation of biological systems in the last decade. Unfortunately, they generally rely on fluorescent labeling for cellular phenotyping, an indirect measure of the molecular landscape in the cell, which has critical limitations. Here we demonstrate Raman image-activated cell sorting by directly probing chemically specific intracellular molecular vibrations via ultrafast multicolor stimulated Raman scattering (SRS) microscopy for cellular phenotyping. Specifically, the technology enables real-time SRS-image-based sorting of single live cells with a throughput of up to ~100 events per second without the need for fluorescent labeling. To show the broad utility of the technology, we show its applicability to diverse cell types and sizes. The technology is highly versatile and holds promise for numerous applications that are previously difficult or undesirable with fluorescence-based technologies.

Intelligent image-activated cell sorting 2.0.

The advent of intelligent image-activated cell sorting (iIACS) has enabled high-throughput intelligent image-based sorting of single live cells from heterogeneous populations. iIACS is an on-chip microfluidic technology that builds on a seamless integration of a high-throughput fluorescence microscope, cell focuser, cell sorter, and deep neural network on a hybrid software-hardware data management architecture, thereby providing the combined merits of optical microscopy, fluorescence-activated cell sorting (FACS), and deep learning. Here we report an iIACS machine that far surpasses the state-of-the-art iIACS machine in system performance in order to expand the range of applications and discoveries enabled by the technology. Specifically, it provides a high throughput of ∼2000 events per second and a high sensitivity of ∼50 molecules of equivalent soluble fluorophores (MESFs), both of which are 20 times superior to those achieved in previous reports. This is made possible by employing (i) an image-sensor-based optomechanical flow imaging method known as virtual-freezing fluorescence imaging and (ii) a real-time intelligent image processor on an 8-PC server equipped with 8 multi-core CPUs and GPUs for intelligent decision-making, in order to significantly boost the imaging performance and computational power of the iIACS machine. We characterize the iIACS machine with fluorescent particles and various cell types and show that the performance of the iIACS machine is close to its achievable design specification. Equipped with the improved capabilities, this new generation of the iIACS technology holds promise for diverse applications in immunology, microbiology, stem cell biology, cancer biology, pathology, and synthetic biology.

Simple Isolation of Single Cell: Thin Glass Microfluidic Device for Observation of Isolated Single Euglena gracilis Cells.

Single cell analysis has gained attention as a means to investigate the heterogeneity of cells and amplify a cell with desired characteristics. However, obtaining a single cell from a large number of cells remains difficult because preparation of single-cell samples relies on conventional techniques such as pipetting that are labor intensive. In this study, we developed a system combining a 0.6-mm thin glass microfluidic device and machine vision approach to isolate single Euglena gracilis cells, as a model of microorganism with mobility, in a small/thin glass chamber. A single E. gracilis cell in a chamber was cultured for 4 days to monitor its multiplication. With this system, we successfully simplified preparation of single cells of interest and determined that it is possible to combine it with other analytical techniques to observe single cells continuously.

Intelligent Image-Activated Cell Sorting.

A fundamental challenge of biology is to understand the vast heterogeneity of cells, particularly how cellular composition, structure, and morphology are linked to cellular physiology. Unfortunately, conventional technologies are limited in uncovering these relations. We present a machine-intelligence technology based on a radically different architecture that realizes real-time image-based intelligent cell sorting at an unprecedented rate. This technology, which we refer to as intelligent image-activated cell sorting, integrates high-throughput cell microscopy, focusing, and sorting on a hybrid software-hardware data-management infrastructure, enabling real-time automated operation for data acquisition, data processing, decision-making, and actuation. We use it to demonstrate real-time sorting of microalgal and blood cells based on intracellular protein localization and cell-cell interaction from large heterogeneous populations for studying photosynthesis and atherothrombosis, respectively. The technology is highly versatile and expected to enable machine-based scientific discovery in biological, pharmaceutical, and medical sciences.

Holographic microinformation hiding.

We propose a holographic microinformation hiding scheme in which the embedding information to be embedded is small and imperceptible to the human eyes. This scheme converts the embedding information into a complex amplitude via scaled diffraction. The complex amplitude of the reduced embedding information is added to the complex amplitude of the host image, followed by conversion to a hologram. The reduced embedded information is inconspicuous from the hologram during the reconstruction process; however, the reduction leads to the degradation of the embedded image quality. Therefore, to improve the quality of the embedded image quality, we employ iterative optimization and the time averaging effect of multiple holograms.

Color computer-generated hologram generation using the random phase-free method and color space conversion.

We propose two calculation methods of generating color computer-generated holograms (CGHs) with the random phase-free method and color space conversion in order to improve the image quality and accelerate the calculation. The random phase-free method improves the image quality in monochrome CGH, but it is not performed in color CGH. We first aimed to improve the image quality of color CGH using the random phase-free method and then to accelerate the color CGH generation with a combination of the random phase-free method and color space conversion method, which accelerates the color CGH calculation due to down-sampling of the color components converted by color space conversion. To overcome the problem of image quality degradation that occurs due to the down-sampling of random phases, the combination of the random phase-free method and color space conversion method improves the quality of reconstructed images and accelerates the color CGH calculation. We demonstrated the effectiveness of the proposed method in simulation, and in this paper discuss its application to lensless zoomable holographic projection.

Random phase-free kinoform for large objects.

We propose a random phase-free kinoform for large objects. When not using the random phase in kinoform calculation, the reconstructed images from the kinoform are heavy degraded, like edge-only preserved images. In addition, the kinoform cannot record an entire object that exceeds the kinoform size because the object light does not widely spread. In order to avoid this degradation and to widely spread the object light, the random phase is applied to the kinoform calculation; however, the reconstructed image is contaminated by speckle noise. In this paper, we overcome this problem by using our random phase-free method and error diffusion method.

Nonuniform sampled scalar diffraction calculation using nonuniform fast Fourier transform.

Scalar diffraction calculations, such as the angular spectrum method (ASM) and Fresnel diffraction, are widely used in the research fields of optics, x rays, electron beams, and ultrasonics. It is possible to accelerate the calculation using fast Fourier transform (FFT); unfortunately, acceleration of the calculation of nonuniform sampled planes is limited due to the property of the FFT that imposes uniform sampling. In addition, it gives rise to wasteful sampling data if we calculate a plane having locally low and high spatial frequencies. In this Letter, we developed nonuniform sampled ASM and Fresnel diffraction to improve the problem using the nonuniform FFT.

Lensless zoomable holographic projection using scaled Fresnel diffraction.

Projectors require a zoom function. This function is generally realized using a zoom lens module composed of many lenses and mechanical parts; however, using a zoom lens module increases the system size and cost, and requires manual operation of the module. Holographic projection is an attractive technique because it inherently requires no lenses, reconstructs images with high contrast and reconstructs color images with one spatial light modulator. In this paper, we demonstrate a lensless zoomable holographic projection. Without using a zoom lens module, this holographic projection realizes the zoom function using a numerical method, called scaled Fresnel diffraction which can calculate diffraction at different sampling rates on a projected image and hologram.

In-line digital holographic microscopy using a consumer scanner.

We demonstrate an in-line digital holographic microscopy using a consumer scanner. The consumer scanner can scan an image with 4,800 dpi. The pixel pitch is approximately 5.29 µm. The system using a consumer scanner has a simple structure, compared with synthetic aperture digital holography using a camera mounted on a two-dimensional moving stage. In this demonstration, we captured an in-line hologram with 23, 602 × 18, 023 pixels (≈0.43 gigapixels). The physical size of the scanned hologram is approximately 124 mm × 95 mm. In addition, to accelerate the reconstruction time of the gigapixel hologram and decrease the amount of memory for the reconstruction, we applied the band-limited double-step Fresnel diffraction to the reconstruction.

Band-limited double-step Fresnel diffraction and its application to computer-generated holograms.

Double-step Fresnel diffraction (DSF) is an efficient diffraction calculation in terms of the amount of usage memory and calculation time. This paper describes band-limited DSF, which will be useful for large computer-generated holograms (CGHs) and gigapixel digital holography, mitigating the aliasing noise of the DSF. As the application, we demonstrate a CGH generation with nearly 8K × 4K pixels from texture and depth maps of a three-dimensional scene captured by a depth camera.

Fast high-resolution computer-generated hologram computation using multiple graphics processing unit cluster system.

To overcome the computational complexity of a computer-generated hologram (CGH), we implement an optimized CGH computation in our multi-graphics processing unit cluster system. Our system can calculate a CGH of 6,400×3,072 pixels from a three-dimensional (3D) object composed of 2,048 points in 55 ms. Furthermore, in the case of a 3D object composed of 4096 points, our system is 553 times faster than a conventional central processing unit (using eight threads).

Generation of real-time large computer generated hologram using wavefront recording method.

We report the generation of a real-time large computer generated hologram (CGH) using the wavefront recording plane (WRP) method with the aid of a graphics processing unit (GPU). The WRP method consists of two steps: the first step calculates a complex amplitude on a WRP that is placed between a 3D object and a CGH, from a three-dimensional (3D) object. The second step obtains a CGH by calculating diffraction from the WRP to the CGH. The disadvantages of the previous WRP method include the inability to record a large three-dimensional object that exceeds the size of the CGH, and the difficulty in implementing to all the steps on a GPU. We improved the WRP method using Shifted-Fresnel diffraction to solve the former problem, and all the steps could be implemented on a GPU. We show optical reconstructions from a 1,980 × 1,080 phase only CGH generated by about 3 × 10(4) object points over 90 frames per second. In other words, the improved method obtained a large CGH with about 6 mega pixels (1,980 × 1,080 × 3) from the object points at the video rate.

Time-division color electroholography using one-chip RGB LED and synchronizing controller.

We propose time-division based color electroholography with a one-chip RGB Light Emitting Diode (LED) and a low-priced synchronizing controller. In electroholography, although color reconstruction methods via time-division have already been proposed, the methods require an LCD with a high refresh rate and output signals from the LCD for synchronizing the RGB reference lights such as laser sources, which consequently increase the development cost. Instead of using such an LCD, the proposed method is capable of using a general LCD panel with a normal refresh rate of 60 Hz. In addition, the LCD panel used in the proposed method does not require the output signals from the LCD. Instead, we generated synchronized signals using an external controller developed by a low-priced one-chip microprocessor, and, use a one-chip RGB LED instead of lasers as the RGB reference lights. The one-chip LED allows us to decrease the development cost and to facilitate optical-axis alignment. Using this method, we observed a multi-color 3D reconstructed movie at a frame rate of 20 Hz.